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^ rom a systems analysis viewpoint, an evaluation system is a subsystem that is 
upenmposed on the educational system to provide information for decisions related 
to its maintenance and goals. Educational outcomes, viewed as a much broader 
concept than the traditional view of changes in pupil behavior, represent the startmq 

terms ^of ^J va,uat,on act,vlt y In education. To evaluate an instructional program in 
of f lts „ s '9mficant interactions, an evaluation strategy (plan) is required 
which systematically provides information concerning both (1) input and output 

of a the e slct USed H V re,ate ? Subsysterns and (2) output changes caused by alterations 
of the system under consideration. (JH) 1 
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THE ROLE OF EDUCATIONAL OUTCOMES 
IN EVALUATION STRATEGY 



Walter J. Foley 



In order to clarify the role of educational outcomes in evaluation strategy, it is 
first necessary to detail what is meant by a strategy. The fact that an evaluation strategy 
is necessary has never been clearer in American education. We are faced with an 
unprecedented challenge-the challenge of providing an alternative to social chaos. Our 
public and our patrons are looking to education as never before to provide stabilization in 
these times of crisis. We are caught on the horns of a dilemma. At the same time that 
technological change has forced us to reexamine the basis for our teaching relationships, 
mass social changes have forced us to consider what we are doing to the students in our 
system— to examine the very process of education as it exists today. 

To better understand the need for an evaluation strategy, let us make a slight 
detour and consider two kinds of administrators that perhaps do not exist but typify 
extreme positions. First, we have the administrator whose behavior can best be summed 
under the phrase "shoot from the hip. n He has been characterized as funning the school 
system from his hip pocket. At the other extreme we have the administrator who is for- 
ever laying out all possible alternatives in every situation and never gets to the point where 
any action is taken. This is the "watch and wait" posture of the man who always needs 
more information. In the terminology of this paper, these two types of behavior are con- 
sidered tactics as contrasted to strategy for neither has a plan. 

Evaluation Strategy 

Strategy is an overall plan designed to achieve a goal. Many situations do not call 
for the application of the strategy. The kinds of situations where strategies are important- 
in fact these days imperative-are those that require decisions across successive periods 
of time. For example, a strategy has the relationship of a game plan in football to the 
tactic of a specific play for a "third down-one yard to go" situation. 

The second condition necessary for developing a strategy has to do with uncertainty. 

Today it is quite apparent that uncertainty is the rule rather than the exception. For this 
reason, it is necessary for educational administrators to develop a strategy. If we were 
certain about the future, we would not need a strategy — we would just need a decision. 

Perhaps the uncertainty principle can be best summed up as a series of "if-then" situa- 
tions. The strategy governs the tactics. 

The last factor to be mentioned in support of the development of a strategy has to 
do with the decrease in uncertainty over time. Education is that kind of situation which 
clearly shows a decrease in alternatives over time. The longer a pupil is in the system, 
the closer he becomes, in terms of possible alternatives, to becoming an outcome and the 
less chance there is for the educational process to affect his behavior. 
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To summarize our concept of a strategy we would say that a strategy is necessary 
when alternatives exist, when a series of decisions that span time have to be made, and 
when the closer one gets to the end of the system there are fewer alternatives than the 
number that existed at the time a pupil entered. To the extent that it is not possible to 
answer questions and provide alternatives over time, it is impossible to formulate strategy 
in a meaningful sense of the word. 

Strategy and Goals 

Since we have defined the conditions which demand a strategy, our next step is to 
relate the concept to educational goals. At the national level the place of strategy is that 
of describing an overall plan for achieving national goals. Recently our nation dedicated 
itself to improving educational opportunity for disadvantaged youth. This goal was opera- 
tionalized by transforming it into two broad strategies. The first consisted of attempts 
to change the relationship between disadvantaged pupils and the school within the school 
setting. This strategy is best known as compensatory education . The other thrust was 
made through the creation of new educational agencies and by involving new departments 
of the government in the educational system. An example would be the creation of the 
Head Start program through the Office of Educational Opportunity. In addition, Vista 
Volunteers, the Job Corps, and Community Action Programs were also initiated. 

Today at the national level we are in a position to make new decisions based on the 
time lapse that has occurred and the evaluation that has been made since the initial pro- 
grams were placed into operation. An example of a shift in tactics would be placing of the 
Head Start Program into the Department of Health, Education and Welfare. This change in 
tactics resulted, we hope, from evidence on the operation of the program since its 
inception. Thus, the providing of informationoon the results of programs created to 
implement a national strategy is viewed in relation to a national goal. This is as it should 
be and is a part of what we have in mind when we talk about evaluation. 

Evaluation Systems 

We are now in a position to discuss evaluation as a concept. Evaluation systems 
are imbedded in educational systems and serve as devices for correcting performance 
toward a real or desired standard. By being sensitive to deviations from the standard and 
feeding the deviations back into the system, evaluation is a process of providing informa- 
tion for decision mak in g in educatinn. 

At this point you have probably noticed that evaluation systems have replaced 
evaluation in the context of educational systems. As the word system is being used here, 
it has certain characteristics. First, a system exists in terms of having physical bound- 
aries and being identifiable. Indeed, a person can perceive it, he can locate it, he can 
comprehend it. Also, a system is what it is- — nothing more, nothing less. By-tm)s we 
mean that a system performs a function and can exist without any relationship to how well 
it performs the function or hew it relates to the world. It is when you add the concept of 
evaluation that a system begins to have restrictions placed upon it. 

A system consists of both men and materials which operate in f un ctio nal relation- 
ships to one another. In other words, the men and materials are interdependent. For 
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w _ e des p r ibe systems in teras of their input, process, and output. 

‘ ^ducationai system from a traditional vantage point, a pupil is considered 

as the input, process is usually viewed as instruction; pupils changed by the process and 

de^ic^b^ffed^n^h T'T ^ ou ^? ut * Pn evaluation system serves as the correction 
f ® ? 211110 ? he educatuonal system discrepancies between the actual output 

SnceptuS^^ P f;i l 13 f6edbaCk proV±3ion which is the h eart of an evaluation 

^valuation, to use a mechanical analogy, is the servo-mechanism for 

spOOifiOOtiOO^? 113 “ b he system in order to produce a product that is consistent with the 
specification of expectation for the product. 

» 

This digression into systems concept was made in order to understand the place 

£“^ Uat r 37S ^®“ in . an Rational system. Also, it has allowed us to derive certain 

satisfSt^ u’ ° Ut F UtS “ay® 4 ®® 3 term2 are never inadequate; given an input that is 
atistactory, the systems that produce these outputs are inadequate. Two, unspecified 

"? not detectable. In addition, unmeasured variations in output that 
“ t0 the system cannot affect the operation of the system. These principles 
^®f“ , the role of ev^-uation. The concept is useful in that it allows us to define and con- 

® FT° ces ? ®°difi®ation withou reference to the exact content and concerns itself 

with alternatives in functional organization * 



, „ f?* u ! summarize. An evaluation system is in reality a subsystem of an 

!w_+ naX system; a subsystem in the sense that it is superimposed upon an educational 
system to provide information for decisions related to the maintenance and goals of the 
em Evaluation, when viewed this way, has moved from the posture of 
determining the ^ worth of A versus B to one that provides information for complex educa- 
tional systems in light of buildings, investments, maintenance, food services, supplies, 
pup 4 s > colters and emerging technology. Evaluation is now in the position 
of providing information for new conclusions regarding the impact and importance of 



Educational Outcome 

With the background of systems concepts in mind, educational outcomes form a 

evaluating our educational programs. We start with the present system 
in terms of what its products look like, what elements in terms of staff, facilities, and 

wf^nw nt i red 221110 the P rocess of explicating the curriculum that produced these products. 
The curriculum serves as the vehicle for making changes in the input of our system, as 

time 38 lnC0rp0rating change ^ the structure and relationship of elements over a period of 



Educational at comes represent the starting point for any evaluation activity in 
education. The practical standard for deviloping a strategy is the present status of the 
output. By expanding the present status of output to include all interrelated information 
and placing this information in juxtaposition with ideal situations, standards, past 
counterparts from other systems, and lacks or gaps in our measurement, we 
will be in a position to provide information systematically to decision makers. From a 
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point of view, the traditional conceptualization of outcome as only pupil behavior 

upon termination is far too limited. This important aspect of outcome will be elaborated 
next. 

levels of Outcome 

By using the same systems reasoning for our conceptualization of outcomes, we 
can better understand the levels problem in education. The outcome-which is hopefully 
'a behavioral change- in a pupil-teacher interaction has been a focus Df concern for 
many years. When a teacher attempts to move from the level of a specific interaction to 
one of providing information on the outcome of a unit of material, the levels problem 
becomes apparent. What are the decision rules? Do I add all positive responses? Do I 
take off for mistakes? Have I asked for what I wish every pupil to know? At the next 
level a teacher puts together the information from the units to form a mark for the period. 
Then the marks for the periods are put together to form a mark for the subject. We 
could go on to subject combinations for producing a standing within grade within area 
within state and on and on. Suffice it to say that data aggregation causes a loss in 
information about the specificity of pupil response potential for given situations. 

The point I would like to make about outcomes is simnly that from a systems 
standpoint, the instructional program is imbedded in a staff system, a facilities system, 
and a financial system, and in reality there are multiple outcomes. Outcomes in 
education consist of much more than bhanges in pupil behavior. For we must realize 
that simultaneous change is also taking place in the other subsystems of our educational 
system. Indeed, the care and maintenance of our educational system must be based upon 
evaluation information that specifies and feeds back a wide variety of oufcctme data. For 
today 1 s evaluation information becomes tomorrowSs management data. 



Evaluation Strategy 

. Since this paper represents an attempt at defining a role for educational outcomes 
in building an evaluation strategy. An since we have presented a case for multiple levels 
of outcomes across interrelated subsystems, it is only fair that our conceptualizations of 
an evaluation strategy be explicated. 

u The criteria for an evaluation system are more than those of validity and reliabil- 
ity of data. For evaluation is more than measurement. To meet the needs of decision 
making, evaluation must also mefet the criteria of relevance, timeliness, credibility, 
persuasiveness and efficiency. The evaluation system of the future will be an information 
system serving decision making. 

Scope of an EvAlnat.-i r»n 

Although they have been previously stated, it is now appropriate to bring together 
the principles which should guide our thinking in examining education from a systems 
point of view. For, contrary to what one would expect, given our experience in the physi- 
cal sciences on the interrelatedness of events, we have traditionally studied the learning 
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Phenomena in eduation with regard for the restrictions caused by the system* s actual 
operation. 

In the proposed evaluation strategy, the goal is to depict the instructional program 
in terms of all its significant interactions. A systems approach deliberately complicates 
events to include all components that significantly contribute to the output. 

Again, this methodology must be related to levels of decisions as they define the 
Scope of the system depicted and the representation must include all interrelationships that 
impinge on the system at that particular level. The meaning of interrelationships in the 
previous sentence is equal to "significantly related." 

The above statement again points to the conceptual necessity of considering system 
input as output of other systems and system output as in input for still other systems. 

Thus, both input and output possess restrictions caused by other systems and change in 
output caused by alteration of the system under consideration must be viewed in the larger 
context of other systems. 

An example from the work of the Iowa Educational Information Center will help 
illustrate these principles. Flexible modular scheduling (a device for implementing 
pupil schedules based upon fifteen minute time modules through the use of a computer) is 
being introduced into several Iowa high schools. 

What was at first blush considered as a way of co mbinin g large group-small group 
instruction and team teaching to better teach subject matter actually turned out to have 
effects on teacher-administrative, teacher-pupil, teacher-parent, pupil-pupil, adminis- 
trator-pupil, parent-administrator and school-community relationships. 

After several years of false starts and conflict situations, the project is now con- 
sidered as a human interaction change process rather than as a curricular tool or a 
budgeting device. We found that flexible scheduling affects the specifications of pupil 
input characteristics, the use of space, the allocation of staff resources, the develop- 
ment of subject matter content and the interpersonal relationship of the school to the 
community, in addition to a hoped for "better way of teaching content 1 ' =■ in other words, 
the installation of flexible scheduling effected systems other than the curriculum system. 

Our goal at the Center is to develop the educational program as the vehicle for 
this new concept of evaluation. A concept based on the provision of information for 
planning and management of resources. We would like you to join us in this work. 
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